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Description 

The present invention generally relates to an on-demand multimedia server and more particularly to a server, 
responsive to a request from one of consuming devices, for serving requested multimedia information in a format 

5 adapted to the requesting consuming device. 

Some of the terms used herein will be first defined for the sake of better understanding the following description. 
Multimedia information or each of multimedia applications is a compiled set of single-media data such as video, ani- 
mation, graphics, sound, text and computer program, which each are called a "material. 0 Each material is represented 
by "material data' contained in a "material file." Information on a specific material (file) including the file name and 

io attributes of the material data is referred to as "material information." The data format of material data or file is referred 
to as "material format," which may be, for example, an MPEG 1 (Moving Picture Coding Experts Group 1) format in 
case of video, a JPEG (Joint Photographic Coding Experts Group) format in case of graphics and so forth. Each of 
multimedia applications generally comprises some kinds of material data constituting the multimedia application and 
■scenario data" which describes how each material plays its role in the application. Information on a specific scenario 

is (file) including the file name and attributes of the scenario is referred to as "scenario information." 

There are various kinds of so-called multimedia service systems. In such a system, each of multimedia applications 
is prepared by preparing material data used in the application and creating a scenario data for the application in a 
multimedia source device and by storing the material data and the scenario data in a multimedia server. Multimedia 
information is provided to the user in a multimedia consuming device or play-back device by the multimedia server 

20 serving material data and scenario data for a specified multimedia application to the consuming device, which uses 
the served data to reproduce or play back the specified multimedia application. In this way, prepared material data and 
scenario data are stored and served as they are. 

As described above, there are various types of materials such as video, animation, graphics, sound, text and 
computer program. There may be also various data formats even for the same type of material. Specifically, data 

25 formats for a material type of, e.g., video (or moving picture) include DV, MPEG 2, MPEG 1 , etc., and there are variety 
of data formats for a material type of, e.g., graphics (or still picture) including JPEG, BMP (the standard bit-mapped 
graphics format used in the Windows, environment), etc. 

If two materials are in different formats even though the materials are of an identical material type, reproduction 
of the two materials stored in the different format needs different play-back devices with respective structures and 

30 functions. To take an example, a play-back device for playing back moving picture data in the DV format has a function 
of extracting an. image signal from DV format data. Similarly, a play-back device for playing back moving picture data 
in the MPEG 1 format has a function of extracting an image signal from MPEG 1 format data. These two functions are 
different from each other and neither function includes the other. 

Further, in order to play back moving picture of a bit rate (which is determined by the frame size, the frame frequency 

35 and the resolution of the moving picture) exceeding a certain level, it is necessary to realize, by means of hardware, 
at least a part of the function of extracting an image signal from moving picture data regardless of whichever format 
the moving picture data is in. For this reason, it is not a practical solution to provide each play-back device with play- 
back functions for different data format. 

It is therefore an aim of the invention to provide a multimedia information server which serves each of consuming 

40 devices (or client play-back devices) with a multimedia application in a format adapted to the consuming device. 

In a multimedia information service system, a server is supplied with multimedia applications each comprising a 
plurality of kinds of materials and a scenario which defines a role of each material The server registers or stores each 
(multimedia) application in response to a reception thereof (registration operation) and, in response to a request from 
one of a plurality of consuming devices or clients, serves a requested one of the applications in a format with which 

45 the requesting client can deal with respect to materials of a certain kind (eg., moving picture materials) among the 
plurality of kinds (service operation). In accordance with one aspect of the invention, the server converts an input (or 
original) format of the certain kind of materials of at least one of the applications into at least one of target formats 
associated with the input format either in registration operation or in service operation (material conversion). Each 
scenario of the at least one application which scenario is for said input format is converted into at least one scenario 

50 of the at least one application which scenario is adapted to at least one of target formats either in registration operation 
or in service operation (scenario conversion). In response to a request for one of the applications, the server serves 
the requesting one of the consuming devices with at least a scenario of the requested application which scenario has 
been adapted to the format with which the requesting consuming device can deal. This enables every consuming 
device to consume any of the multimedia applications stored in the server regardless of the material format with which 

55 the consuming device can deal with respective to a certain kind of materials. 

In one specific embodiment of the invention, both of the material conversion and the scenario conversion are 
performed in registration operation. This enables a quick response in service operation but will need a mass storage 
device of a largest capacity. 
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In another specific embodiment of the invention, the material conversion which takes more time than the scenario 
conversion is performed in registration operation and the scenario conversion is performed in service operation. This 
enables a relatively quick response in service operation with a mass storage device of a relatively targe capacity. 

In further specific embodiment of the invention, both of the material conversion and the scenario conversion are 
s performed in service operation. This can save the capacity of the mass storage device but will cause the service 
operation to take time. 

In still further specific embodiment of the invention, materials which take more time for format conversion are 
converted in the registration operation, and the scenario of requested application and, if the materials for the requested 
application have not been converted, such materials are converted in the service operation. In this case, the server 
10 will afford a quicker service at the cost of moderate capacity of the mass storage device. 

The input material formats maybe limited to master data formats, i.e., data formats that each contains the maximum 
quantity of information in all the data formats for one category of materials, and material formats available for service 
may be obtained by converting the master data formats. By doing this, the server can provide multimedia information 
offurther higher quality. 

15 Further features and advantages of the present invention will be apparent from the following description of exem- 

plary embodiments and the accompanying drawings in which: 

FIG. 1 is a schematic block diagram showing an exemplary multimedia information service system using a multi- 
media information server to which the principles of the invention is applied; 
20 FIG. 2 shows an exemplary structure of scenario data for an application using moving pictures of the DV format 

as materials; 

FIG. 3 is a flow chart showing an exemplary registration operation executed by the multimedia server 10 under 
the control of a program to which scheme 1 of the invention is applied; 

FIG. 4 is a material table which contains material IDs and material information for all the materials having been 
25 registered so far in the server 10; 

FIG. 5 is an exemplary structure of a scenario which has been obtained by replacing MatlDinVSC's 221 with 
corresponding material IDs (Matl Din Server's) (401 ) in accordance wilh the table 400 of FIG. 4 and eliminating the 
material information (MatData) in the scenarb 200; 

FIG. 6 is an exemplary available material format table which comprises an input format field and an output (or 
30 target) format field which contains target formats into which the input target format is to be converted; 

FIG. 7 is an exemplary scenario conversion table each record of which contains scenario conversion information 
or data to be converted with the format conversion of materials for each of possible conversion patterns (i.e., 
combinations of possible input formats and output (or target) formats); 

FIG. 8 is a scenario 800 obtained by converting data in the ID-converted scenario which data are dependent on 
35 the material format referring to the scenario conversion table of FIG. 7; 

FIG. 9 is a diagram showing a part of a scenario information table used for identifying the scenarios in the server 1 0; 
FIG. 10 is a flow chart showing an exemplary service operation executed by the multimedia server 10 under the 
control of a service program to which scheme 1 of the invention is applied; 

FIG. 11 is a scenario obtained by rewriting the material IDs and adding material information (MatData) to each line 
40 of the MATERIAL section in a scenario read from the storage device 11 in response to a service request; 

FIG. 12 is a flow chart showing an exemplary registration operation executed by the muftimedia server 10 under 
the control of a program to which scheme 2 of the invention is applied; ^ 

FIG. 1 3 is a diagram showing a part of a scenario information table used for identifying the scenarios in the server 
10; 

45 FIG. 14 is a flow chart showing an exemplary service operation according to scheme 2 of the invention; 

FIG. 15 is a flow chart showing an exemplary registration operation according to scheme 3 of the invention; 

FIG. 16 is a flow chart showing an exemplary service operation according to scheme 3 of the invention; 

FIG. 17 is an exemplary available material format table which defines and classifies possible format conversions 

by operation in which the conversion is to be done; 
50 FIG. 18 is a flow chart showing an exemplary registration operation according to scheme 4 of the invention; 

FIG. 19 is a table the server 10 keeps to manage the materials in the server 10; 

FIG. 20 is an example of a scenario stored as a result ofthe steps 181 and 182; 

FIG. 21 is a flow chart showing an exemplary service operation according to scheme 4 of the invention; and 
FIG. 22 is an exemplary master data table used in embodying a fifth scheme of the invention. 

55 

Throughout the drawing, the same elements when shown in more than one figure are designated by the same 
reference numerals. 

FIG. 1 is an exemplary multimedia information service system 1 using a multimedia information server 10 to which 
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the principles of the invention is applied. The multimedia information service system 1 comprises a multimedia source 
device 2 for preparing and supplying multimedia applications each comprising scenario data and material data, the 
multimedia information server 10 which stores the multimedia applications supplied from the source device 2 and 
serves them in a format associated with the requesting device, a plurality of remote consuming devices or client devices 
5 3 with a function of obtaining one of the multimedia applications from the server 10 and playing back the obtained 
multimedia application and transmission media 4 which connects the multimedia server 1 0 with the consuming devices 
3 served by the server 10. 

The multimedia server 10 may be any suitable computer provided with a mass storage device 11, which stores 
programs for controlling the server 10 and all of application data served by the server 10. The consuming device or 
10 client devices 3 may be any device having the above described function such as various tuners, play-back devices, 
multimedia information terminals, PCs (personal computers), etc. 

In operation, the multimedia server 10 does two main jobs, that is, a registration of multimedia application (a 
registration operation) which is initiated by the supply of the multimedia application from the source device 2 and a 
service of multimedia application (a service operation) which is initiated by a request from one of the consuming devices 
ts 3 served by the server 1 0. 

In order to achieve the object of the present invention, the server 10 has only to convert the scenario and the 
materials of a supplied multimedia application at any time in a time period from the reception of the multimedia appli- 
cation to the service thereof. Thus, we will propose four conversion schemes different in whether the conversions of 
the scenario and the materials are executed in the registration operation and/or the service operation as shown in the 
20 following table. 



Table 



Scheme 


Registration 


Service 


1 


Scenario, Materials 




2 


Materials 


Scenario 


3 




Scenario, Materials 


4 


Time consuming data 


Time saving data 



If the scenario and the materials of each multimedia application are converted in the registration operation for the 
application (scheme 1 in the above table), the server 10 will afford a quickest service but will need a mass storage 
device 11 of largest capacity. If the materials of each application are converted in the registration operation for the 
application and the scenario of requested application is converted in the service operation (scheme 2), the server 10 
will afford a quicker service at the cost of moderate capacity of the mass storage device 11. If both the scenario and 
the materials are converted in the service operation (scheme 3), the server 10 will only need the smallest capacity for 
the mass storage device 11 but will be slowest in response. In scheme 4, materials which take more time for format 
conversion are converted in the registration operation, and the scenario of requested application and, if the materials 
for the requested application have not been converted, such materials*are converted in the service operation. In this 
case, the server 10 will afford a quicker service at the cost of moderate capacity of the mass storage device 11 . 

In any of above mentioned schemes, the registration operation is initiatetfby the server 10 receiving a multimedia 
application (i.e., a set of materials and a scenario in which the roles the materials should play are described) from the 
multimedia source device 2. FIG. 2 is an example of scenario data for an application using moving pictures of the DV 
format as materials. 

In FIG. 2, the scenario (data) 200 comprises a TITLE section 210, a MATERIAL section 220 where information on 
the materials used in the application is described, and an EVENT section 230 where how the materials listed in the 
MATERIAL section 220 act in the application is described. In the TITLE section 210, an item D "ntleName" 211 indicates 
the title or name of the application, e.g., TitleOOOl in this specific example. 

In the MATERIAL section 220, each of the materials is described in a single line. In each line, an item "MatlDinVSC 
221 indicates the identifier of the material described in the line which identifier is valid only in the scenario 200, an item 
MatFormat 222 indicates the material format of the material, and an item MatData 223 indicates material information 
on the material (or the file name of the material). As seen from this section, the multimedia application in this example 
comprises three materials with titles of V0001, V0002 and V0003; the material V0001 is in the DV format and has 
material information of "movieOOOl .dv"; the material V0002 is in the DV format and has material information of 
a movie0002.dv B ; and the material V0003 is in the BMP format and has material information of "movie0003.bmp 0 . 

In the EVENT section 230, each of the evens is described in a single line. Each line comprises an item EventID 
231 which indicates an identifier of an event described by the line which identifier is valid only in the scenario 200, an 
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item Time 232 which indicates the time to hold the event, the above mentioned item MatlDinVSC 221 , an item Location 
234 which indicates, in coordinates, a position on the screen where the event is held, i.e., the material for the event is 
displayed or played, and an item Action 235 which indicates an action to take for the event. In this specific example, 
the multimedia application comprises four events with event IDs (identifiers) of E0001, E0002, E0003 and E0004. As 

5 event E0001 , the material V0001 is played at the position (300, 400) on the screen at the time of 200; as event E0002, 
the material V0002 is played at the position (200, 300) on the screen at the time of 300; as event E0003, the material 
V0003 is displayed at the position (100, 300) on the screen at the time of 400; and as event E0004, the play of the 
material V0001 being played at the position (300, 400) on the screen is stopped at the time of 400. 

If the server 10 receives a multimedia application from the multimedia source device 2, the server 10 enters the 

10 registration operation. 

Scheme 1 

FIG. 3 is a flow chart showing an exemplary registration operation executed by the multimedia server 10 under 

15 the control of a program to which scheme 1 of the invention is applied. On entering the registration, the server 10 first 
registers or stores the received multimedia application without converting the formats of any materials constituting the 
application in steps 301 and 302. Specifically, in step 301 , the server 1 0 assigns the received scenario a title ID (e.g., 
T0001), assigns the received materials "movieOOOl", "movieOOOl" and "image0001° respective material IDs (say, 
S5117, S5290 and S5116) as shown in FIG. 4, and store the materials S5117, S5290 and S5116 in the mass storage 

20 device 11 . The title ID is an identifier for identifying an application (or a group of scenarios for the application and for 
various material formats) in the server 10 and is used in combination with a material format to identify a scenario for 
the material format. FIG. 4 is a material table 400 which contains material IDs 401 and material information 223 for ail 
the materials having been registered so far in the server 10 (Materials V0001 and V0002 of the MPEG 1 and MPEG 
2 formats are also shown as registered in the material table 400). The material IDs 401 are identifiers which the server 

25 -jo uses to identify the materials. In step 302, the server 10 rewrites material information (MatData) 223 and MatlDin- 
VSC's 221 with corresponding material IDs in the scenario data 200 and stores the scenario data 200 in the mass 
storage device 11 associating the data 200 with the assigned title ID (T0001 in this example) and the moving picture 
material format (the DV format in this example). Doing this permits the server 10 to also identify the materials appearing 
in the scenario by means of the material IDs 401 . FIG. 5 is the scenario which has been written and stored in the mass 

30 storage device 11 in step 302. In the scenario 500, MatlDinVSC's V0001 , V0002 and V0003 (221) have been replaced 
with material IDs (MatlDinServer's) S5117, S5290 and S5116 (401), respectively, in accordance with the table 400 of 
FIG. 4 and the material information (MatData) has been eliminated. 

In the next step 303, the server 10 obtains the material information 223 and the material format 222 of materials 
which needs format conversion (e.g., materials for moving pictures) from the received scenario and further obtains 

35 target formats into which the obtained material format 222 can be converted referring to an available material format 
table as shown in FIG. 6. The table of FIG. 6 comprises an input format field and an output (or target) format field which 
contains target formats into which the input target format is to be converted. In this example, the server 10 obtains, for 
moving picture materials V0001 and V0002, "DV as the material format 222 and "movieOOOl .dv" and "movte0002.dv" 
as material information 223, and learns from the available material format table of FIG. 6 that the material format of 

40 the materials "movieOOOl .dv" and - B movie0002.dv" is to be converted from the DV format to the MPEG 1 and MPEG 2 
formats (and if the materials are of the MPEG 2 format, the server 10 will (earn that the materials are to be converted 
into the MPEG 1 format). * 

Then, the server 10 converts the material format (i.e., the DV format in this example) of each of the materials into 
one of the available target formats, e.g., the MPEG 1 format in step 304; and assigns the converted materials, i.e., 

45 movieOOOl. mpl and movie0002.mp1 respective material IDs, e.g., S2560 and S2737 (401) as shown in FIG. 4 and 
stores the materials in step 305. In order to permit the server 10 to identify the materials both written in the scenario 
and stored in the mass storage device by using only the material IDs 401, the server 10 again rewrites material infor- 
mation and MatlDinVSC's with corresponding material IDs in the received scenario 200 in step 306 to obtain an ID- 
converted scenario whose data structure is the same as that of the scenario 500. 

50 in the next step 307, the server 10 converts data in the ID-converted scenario which data are dependent on the 

material format referring to a scenario conversion table as shown in FIG. 7. The scenario conversion table of FIG. 7 
contains scenario conversion information or data to be converted with the format conversion of materials for each of 
possible conversion patterns (i.e., combinations of possible input formats and output (or target) formats). If, for example, 
a material of the DV format is converted into the MPEG 1 format, the scenario conversion information field in this case 

55 reads "Time = Time*100" in FIG. 7. This means that converting each of the values of the "Time" items 232 into a 
hundredth of the value ensures that a material the format of which has been converted from the DV format to the MPEG 
1 format is played in the same way as the material had not been converted. 

Thus if the material format is converted, then scenario data has to be converted accordingly for the following 



5 



EP 0 864 990 A2 



reason. A multimedia application used in the embodiment of the invention is in the form of a combination of main 
materials of moving pictures and other materials. The main moving picture materials determine the time axis in a play 
of the application, that is, each frame of the main moving picture materials^serves as a unit of time in a application play. 
For this, the unit of time information in a scenario depends on the format of the main moving picture materials. 

s FIG. 8 is a scenario 800 obtained by converting data in the ID-converted scenario which data are dependent on 

the material format referring to the scenario conversion table of FIG. 7. In the scenario 800, MatlDinVSC's 221, i.e., 
V0001, V0002 and V0003 have been replaced with corresponding material IDs 401, i.e., S2560, S2737 and S5116, 
respectively as described in step 306 and the values of the Time items 832 have been scaled down to a hundredth as 
described in step 307. It is noted that the material information (MatData's) 223 have been eliminated in the scenario 

10 800. This is because the material IDs 401 are so determined that the server 1 0 can identify the materials only by means 
of the material IDs. 

In step 308, the server 10 stores the obtained scenario 800 in association with the above mentioned title ID and 
the current target material format. Then the server 10 makes a test in decision step 309 to see if the possible target 
formats obtained in step 303 have been exhausted. If so, then the server 10 ends the registration operation. Otherwise, 

15 the server 10 returns to step 304. 

In this way, scenarios and materials used for the scenarios are prepared and stored for each of the available 
moving picture material format. All of the stored scenarios are managed by using a table of FIG. 9. FIG. 9 is a diagram 
showing a part of a scenario information table used for identifying the scenarios in the server 10. The table of FIG. 9 
comprises the fields of the title ID 901 for identifying multimedia applications, the material format 222 which contains 

20 available material formats for the application, and scenario information 902 which is a file name of a file containing the 
scenario having the title ID and the material format. As seen from FIG. 9, the application identified by the title ID TOOOI " 
is available in any of the three material formats, i.e., the DV, MPEG 1 and MPEG 2 formats through the files of scenario 
information title001dv.vsc, title001m1.vsc and title001m2.vsc, respectively. 

FIG. 10 is a flow chart showing an exemplary service operation executed by the multimedia server 10 under the 

25 control of a service program to which scheme 1 of the invention is applied. On receiving, from one of the consuming 
devices or clients 3, a service request for a title ID with a material format of moving picture materials the requesting 
client can play, the server 10 starts the service operation of FIG. 10. It is assumed that the server 10 have received 
T0001 and the MPEG 1 format as the title ID and the moving picture material format, respectively. Then in FIG. 10, 
the server 10 first reads the scenario data identified by the received title ID T0001 and the material format MPEG 1 

30 from the storage device 11 in step 101. 

In step 102, the server 10 rewrite the material IDs with MatlDinVSC's and adds material information (MatData) to 
each line of the MATERIAL section in the read scenario so that the consumer device 3 can identify the data stored in 
the server 10. FIG. 11 is a diagram showing a scenario data obtained by the operation of step 102. Finally in step 103 
the server 10 sends the scenario of FIG. 11 and the material data listed in the MATERIAL section of the scenario to 

35 the requesting client 3. 

Instead of the server 10 sending both of the scenario and the materials in step 103, the server 10 may send the 
scenario only in step 103, and thereafter the client 3 may obtain the materials referring to the received scenario. 

Scheme 2 

40 

In scheme 2, the materials of each application are converted in the registration operation for the application and 
the scenario of a requested application is converted in the service operation .^The server 10 affords a quicker service 
at the cost of moderate capacity of the mass storage device 11 . 

FIG. 12 is a flow chart showing an exemplary registration operation executed by the multimedia server 10 under 

45 the control of a program to which scheme 2 of the invention is applied. In the following description, it is assumed that 
the server 10 receives the same application from the multimedia source device 2 as in the above description of scheme 
1 . The registration operation of FIG. 1 2 is identical to that of FIG. 3 except that the steps 306 through 308 (the registration 
of scenarios for material formats other than the input format, i.e., the DV format in this example) have been eliminated 
and the step 302 has been replaced with step 302a. For this, only the step 302a will be described. 

so in scheme 1 , the scenarios in the server 10 has to be identified by using both of the title ID and the moving picture 

material format in scheme 1 because a plurality of scenarios of different formats are stored for each application, while 
the scenarios in the server 10 of scheme 2 can be identified only by the title ID. Because only the scenario of the input 
format is stored for each application (and accordingly the title ID serves as a scenario ID) in scheme 2. For this reason, 
in step 302a, the server 10 rewrites material information and MatlDinVSC's with corresponding material IDs in the 

55 scenario data; and store the scenario data associating the data with the title ID assigned in step 301 . 

FIG. 1 3 is a diagram showing a part of a scenario information table used for identifying the scenarios in the server 
1 0 according to scheme 2 of the invention. This table corresponds to the table of FIG. 9. However, from just described 
reason, the table of FIG. 1 3 contains no information on material format, that is, the table lacks the MATERIAL FORMAT 
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field and the values of the SCENARIO INFORMATION fields do not include any element indicative of the material 

format as expressed like °title001 .vsc\ 

FIG. 14 is a flow chart showing an exemplary service operation executed by the multimedia server 10 under the 

control of a service program to which scheme 2 of the invention is applied. The service operation of FIG. 14 is identical 
s to that of FIG. 10 or scheme 1 except that the step 101 has been replaced with step 101a and step 307 of FIG. 3 has 

been inserted between the steps 102 and 103. 

On receiving, from one of the consuming devices or clients 3, a service request for a title ID with a material format 

of moving picture materials the requesting client can play, the server 10 starts the service operation of FIG. 14. The 

server 10 first reads the scenario data from the storage device 11 by using the received title ID in step 101a. In this 
10 step, the material format of moving picture materials is not used for the same reason as described in conjunction with 

the step 302a. Then, the server 10 executes the step 102 and proceeds to step 307. In step 307, the server 10 converts 

scenario data dependent on the materiafformat according to the scenario conversion table of FIG. 7. Finally, the server 

1 0 sends the scenario and the materials used in the scenario to the requesting client 3 in step 1 03 to end the registration 

operation. 

15 According to this scheme, since the applications stored in the server 1 0 have had moving picture materials thereof 

format-converted, the server 10 affords a quick response to a service request. 

Scheme 3 

20 in scheme 3, both of the scenario and the materials for a requested application are converted in the service op- 

eration. The server 10 of this scheme only needs the smallest capacity for the mass storage device 11 but is slowest 
in response among the four schemes. 

FIG. 15 is a flow chart showing an exemplary registration operation executed by the multimedia server 10 under 
the control of a program to which scheme 3 of the invention is applied. The registration operation of FIG. 15 is identical 

25 to that of FIG. 1 2 or scheme 2 except that the operation of FIG. 1 5 comprises only two steps 301 and 302a. Specifically, 
the scenario and the materials received from the multimedia source device 2 are so stored that the server 10 can 
manage the scenario and the materials. 

FIG. 16 is a flow chart showing an exemplary service operation executed by the multimedia server 10 under the 
control of a service program to which scheme 3 of the invention is applied. The service operation of FIG. 16 is identical 

30 to that of FIG. 1 4 or scheme 2 except that step 304a (or a material format conversion step) has been inserted between 
the steps 101a and 102. 

After reading the scenario with a title ID specified by the requesting client 3 in step 101a, the server 10 converts 
the material format of each of the materials in the read scenario into the format specified by the client 3 in step 304a. 
Thereafter, the server 10 executes the steps 102, 307 and 103 as in case of FIG. 14 and ends the service operation. 

35 

Scheme 4 

In scheme 4, materials which take more time for format conversion are converted in the registration operation, and 
the scenario of requested application and, if the materials for the requested application have not been converted, such 

40 materials are converted in the service operation. 

FIG. 17 is an exemplary available material format table 170 which defines and classifies possible format conver- 
sions by operation in which the conversion is to be done. In the table 170 of FIG. 17, each record of which comprises 
an INPUT FORMAT field and a TARGET FORMAT field which contains possible target formats into which the input 
format can be converted and which is divided into a CONVERTED AT REGISTRATION TIME field and a CONVERTED 

45 AT SERVICE TIME field. According to the exemplary table 170, if the input format is DV, then the DV format (i.e., 
moving picture materials in the DV format) is to be converted into the MPEG 1 format and the MPEG 2 format in service 
operation and registration operation, respectively. If the input format is MPEG 2, then the MPEG 2 format is to be 
converted into the MPEG 1 format in service operation. 

FIG. 18 is a flow chart showing an exemplary registration operation executed by the multimedia server 10 under 

50 the control of a program to which scheme 4 of the invention is applied. It is assumed in the following description that 
the server 10 have received the same multimedia application as in case of the above described schemes, that is, the 
application with a title TitleOOOl 0 and of the DV format. 

On entering the registration operation of FIG. 18, the server 10 assign the received scenario a title ID; assign the 
received materials respective material group IDs, which each is to be shared by all the materials obtained by converting 

55 the format of a material received from the multimedia source device 2; and store the materials associating the materials 
with respective material group IDs and material formats in step 181. It should be noted that each of the material group 
IDs assigned to the received materials is also assigned to a group of materials into which a received material is con- 
verted in format. For this reason, the server 10 has to use a material group ID and a material format in order to identify 
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a material stored in the mass storage device 11. 

FIG. 19 is a table the server 10 keeps to manage the materials in the server 10. In FIG. 19, each record comprises 

a material group ID (M.G. ID) field containing a material group ID, a MATERIAL FORMAT field containing a material 

format, and a MATERIAL INFORMATION field containing material information or a file name of the material identified 
5 by the material ID and the material format. It is seen from FIG. 2 and 19 that the server 10 assigns a material group 

ID "S0245" to a received material "movieOOOl .aV of the DV format, "S01 40" to "movie0002.oV of the DV format, and 

"S0221 ■ to "imageOOOl .bmp" of the BMP format in step 181 . 

In step 182, the server 10 rewrites material information and MatlDinVSC's with in the scenario data, i.e., replaces 

MatlDinVSC's with corresponding material group IDs and deletes material information; and store the scenario data 
10 associating the data with the assigned title ID. Executing the steps 181 and 182 causes the scenario 200 of FIG. 2 to 

be converted as shown in FIG. 20. In FIG. 20, MatlDinVSC's V0001 , V0002 and V0003 has been replaced with material 

group IDs S0245, S0140 and S0221, respectively, and material information has been eliminated. 

In step 183, the server refers to the table 170 of FIG. 17 and makes a test to see if the input format (or the moving 

picture material format) of the received application is to be converted at the service operation or makes a reverse test 
J5 to see if the materials may have to be converted now, that is, if any of the records whose IN PUT FORMAT fields contain 

the format of the received application has the value of m registration B in the TO-BE -CON VERTED-AT field. If the test 

result is YES or the reverse test result is NO in the decision step 183, then the server 1 0 ends the registration operation. 

(If the input format is DV for example, then the test result is NO and the reverse test result is YES; if the input format 

is MPEG 2 for example, then the test result is YES and the reverse test result is NO.) Otherwise (i.e., the input format 
20 was DV), the server 10 proceeds to step 1 84 and obtains a target format into which the materials are to be converted 

at the registration time from the table of FIG. 17. As seen from the table 170, the server 10 obtains the MPEG 2 format 

as a target format. 

In step 185, the server 10 converts the material format of each of the materials of the received application into the 
target format or MPEG 2. In step 186, the server 10 assigns the converted materials respective material IDs; and store 

2S the materials. Then in decision step 1 87, the server 10 makes a test to see if the target formats into which the materials 
are to be converted now have been exhausted. If so, the server 10 ends the registration operation. Otherwise, the 
server 10 returns to the step 184. 

FIG. 21 is a flow chart showing an exemplary service operation executed by the multimedia server 10 under the 
control of a service program to which scheme 4 of the invention is applied. The service operation of FIG. 21 is identical 

30 to that of FIG. 16 or scheme 3 except that decision step 210 has been inserted between the steps 101a and 304a 
Specifically, after reading out the scenario associated with a specified title ID in step 101a, the server 10 proceeds to 
step 210, where it refers to the table of FIG. 17 and makes a test to see if the material format specified by the client 3 
has been prepared in the registration operation. If so, the server 10 proceeds to the step 102 skipping the step 304a. 
Otherwise, the server 10 proceeds to the step 304a. Operation after the step 304a is identical to that of FIG. 16 or 

35 scheme 3. 

According to scheme 4 of the invention, controlling the timing of format conversion by using the table of FIG. 17 
enables the server 10 to afford quicker service of multimedia applications while preventing the pressure on the mass 
storage device 1 1 . 

40 Scheme 5 

Though in the above described illustrative embodiments, target formats ot the moving picture materials have been 
obtained from various material formats which contain more information as compared with the target formats, it is pref- 
erable to obtain every target format from one of master data formats, i.e., data'formats that each contain the maximum 
45 quantity of information in all the data formats for one category of materials. This enables the server 10 to provide 
multimedia information of much higher quality. As a master data format for moving picture materials, the DV format is 
preferable at present. 

This feature of the invention can be easily realized by any of the embodiments of the above described schemes 
1 through 4 further comprising a master data table as shown in FIG. 22, removing other records than that of a master 

50 data format from the available material format table of FIG. 6 or FIG. 1 7, and, if a received material from the multimedia 
information source is not of the master data format, using a corresponding material of the master data format instead 
of the received material referring to the master data table of FIG. 22 in a format conversion, which is performed in step 
304 of FIG. 3 or 12, in step 304a of FIG. 16, or in steps 185 of FIG. 18 and 304a of FIG. 21. The master data table of 
FIG. 22 contains records for materials stored in the server. Each of the record of the master data table comprises the 

55 fields of material information, master data information from which the material has been obtained, and a master data 
format. 

According to the fifth scheme of the invention, if the server 10 receives an application including a moving picture 
material whose format is of other than a master data format from a multimedia information source without a capability 
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of plating a material of the master data format and if the server 10 has obtained a corresponding material of the master 
data format, then the server 10 can convert in format the application by using the material of the master data format. 

Modifications 

5 

Though format conversions are applied only to the moving picture materials in the above embodiments, format 
conversions may be applied to materials of any other kinds than moving pictures, for example, still picture materials, 
text materials, etc. in the same way. Also, as format conversions, a material format may be converted into one of a 
different resolution or of a different number of colors in the same way. 
10 The moving picture material format the requesting client 3 can play is sent to the server 10 with a title ID in the 

above described embodiments. However, the client 3 may send the client ID instead of the material format and the 
server 10 may keep a table which associates each of the client IDs with a material format the client of the client ID can 
play. 

Many widely different embodiments of the present invention may be constructed without departing from the spirit 
is and scope of the present invention. It should be understood that the present invention is not limited to the specific 
embodiments described in the specification, except as defined in the appended claims. 



Claims 

20 

1. A server for use in a multimedia information service system wherein multimedia applications each comprising 
plural kinds of materials amd a scenario which defines a role of each material are supplied by a source device and 
used by a plurality of consuming devices, a server for serving one of said (multimedia) applications in a format 
with which a requesting one of the consuming devices can deal with respect to materials of a certain kind among 

25 the plural kinds, the server comprising: 

means for converting an input format of said certain kind of materials of at least one of said applications into 
at least one of target formats associated with said input format; 

means, operative on each scenario of said at least one of said applications which scenario is for said input 
30 format, for preparing at least one scenario of said at least one of said applications which scenario is adapted 

to said at least one oftarget formats; and 

means responsive to a request for said one of said applications for serving said requesting one of said con- 
suming devices with at least a scenario of said one of said applications which scenario has been adapted to 
said format with which said requesting consuming device can deal. 

35 

2. A server as defined in claim 1 , wherein said means for converting an input format comprises: 

means, responsive to a reception of a new application from said source device, for converting said input format 
of said certain kind of materials of said new application into all of said target formats associated with said input 
40 format, and said means for preparing at least one scenario comprises: 

means, responsive to said reception and operative on a scenario ofjsaid new application, for preparing sce- 
narios, adapted to ail of said target formats, of said new application. 

3. A server as defined in claim 1 , wherein said means for converting an input format comprises: 

45 

means, responsive to a reception of a new application from said source device, for converting said input format 
of said certain kind of materials of said new application into all of said target formats associated with said input 
format, and said means for preparing at least one scenario comprises: 

means, responsive to a request for said one of said applications and operative on a scenario of said one of 
so said applications which scenario is for said input format, for preparing a scenario of said one of said applications 

which scenario is adapted to said format with which said requesting consuming device can deal. 

4. A server as defined in claim 1 , wherein said means for converting an input format comprises: 

55 means, responsive to a request for said one of said applications, for converting said input format of said certain 

kind of materials of said one of said applications into said format with which said requesting consuming device 
can deal, and said means for preparing at least one scenario comprises: 

means, responsive to said request and operative on a scenario of said one of said applications which scenario 
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is for said input format, for preparing a scenario of said one of said applications which scenario is adapted to 
said format with which said requesting consuming device can deal. 

5. A server as defined in claim 1 , wherein said means for converting an input format comprises: 

5 

means, responsive to a reception of a new application from said source device, for making a first decision on 
whether to postpone conversions of said input format of said certain kind of materials of said new application 
until a request for said new application; 

means responsive to a negative result of said first decision for converting said input format of said certain kind 
10 of materials of said new application into only predetermined ones of said target formats associated with said 

input format; 

means responsive to a request for said one of said applications for making a second decision on whether said 
format with which said requesting consuming device can deal has been prepared for materials of said one of 
said applications; and 

is means responsive to a negative result of said second decision for converting said input format of said certain 

kind of materials of said one of said applications into said format with which said requesting consuming device 
can deal, and said means for preparing at least one scenario comprises: 

means, responsive to said request for said one of said applications and operative on a scenario of said one 
of said applications which scenario is for said input format, for preparing a scenario of said one of said appli- 
20 cations which scenario is adapted to said format with which said requesting consuming device can deal. 

6. A server as defined in claim 1 , wherein said means for converting an input format of said certain kind of materials 
includes means for associating each of possible input formats of said certain kind of materials with target formats 
into which said each of possible input formats can be converted. 

25 

7. A server as defined in any of the preceding claims, wherein a scenario of each of said multimedia applications 
comprises : 

a material description of each of materials constituting said multimedia application, each material description 
30 including format data indicative of a format of a material for which said each material description is intended; 

and 

an event description of each of events controlling said application, each event description including 



35 



8. A server as defined in claim 7, wherein means for preparing at least one scenario comprises: 



means operative on said each material description for converting said format data from said input format to 
said at least one of target formats to yield said at least one scenario for said at least one of target formats; and 
means operative on each event description for rewriting said data dependent on said format by using infor- 
mation associated with said input format and said at least one of target formats to yield said at least one 
40 scenario for said at least one of target formats. 

9. A server as defined in any one of the preceding claims, wherein insteatfof converting said input format of said 
certain kind of materials, said input format converting means converts materials of a master data format from which 
said certain kind of materials of said input format have been obtained. 

45 

10. A server as defined in claim 9, wherein said master data format is a DV format. 

11. A server as defined in any one of the preceding claims, wherein said certain kind is one of moving picture, still 
picture, text, etc. 

so 

12. A server as defined in any one of the preceding claims, wherein said input format includes a certain frequency, 
and said target formats include frequencies different from one another and said certain frequency. 

13. A server as defined in any one of the preceding claims wherein said input format includes a certain number of 
55 displayed colors, and said target formats include numbers of displayed colors which numbers are different from 

one another and said certain number. 

14. A method, for use in a multimedia server which a source device supplies with multimedia applications each com- 
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prising plural kinds of materials and a scenario which defines a role of each material and which serves a plurality 
of consuming devices, for serving one of said multimedia applications in a format with which a requesting one of 
the consuming devices can deal with respect to materials of a certain kind among the plural kinds, the method 
comprising the steps of: 

converting an input format of said certain kind of materials of at least one of said applications into at least one 
of target formats associated with said input format; 

on the basis of each scenario of said at least one of said applications which scenario is for said input format 
and preparing at least one scenario of said at least one of said applications which scenario is adapted to said 
at least one of target formats; and 

in response to a request for said one of said applications, serving said requesting one of said consuming 
devices with at least a scenario of said one of said applications which scenario has been adapted to said 
format with which said requesting consuming device can deal. 

is 15. A method as defined in claim 14, wherein said step of converting an input format comprises the steps of: 

in response to a reception of a new application from said source device, converting said input format of said 
certain kind of materials of said new application into all of said target formats associated with said input format, 
and said step of preparing at least one scenario comprises the step of: 
20 in response to said reception and on the basis of a scenario of said new application, preparing scenarios of 

said new application which scenarios are adapted to all of said target formats. 

16. A method as defined in claim 14, wherein said step of converting an input format comprises the step of: 

in response to a reception of a new application from said source device, converting said input format of said 
certain kind of materials of said new application into all of said target formats associated with said input format, 
and said step of preparing at least one scenario comprises the step of: 

in response to a request for said one of said applications and on the basis of a scenario of said one of said 
applications which scenario is for said input format, preparing a scenario of said one of said applications which 
scenario is adapted to said format with which said requesting consuming device can deal. 

17. A method as defined in claim 14, wherein said step of converting an input format comprises the step of: 

in response to a request for said one of said applications, converting said input format of said certain kind of 
materials of said one of said applications into said format with which said requesting consuming device can 
deal, and said step of preparing at least one scenario comprises the step of: 

in response to said request and on the basis of a scenario of said one of said applications which scenario is 
for said input format, preparing a scenario of said one of said applications which scenario is adapted to said 
format with which said requesting consuming device can deal. ^ 

18. A method as defined in claim 14, wherein said step of converting an input format comprises the steps of: 

in response to a reception of a new application from said source device, making a first decision on whether 
to postpone conversions of said input format of said certain kind of materials of said new application until a 
45 request for said new application; 

in response to a negative result of said first decision, converting said input format of said certain kind of ma- 
terials of said new application into only predetermined ones of said target formats associated with said input 
format; 

in response to a request for said one of said applications, making a second decision on whether said format 
so with which said requesting consuming device can deal has been prepared for materials of said one of said 

applications; and 

in response to a negative result of said second decision, converting said input format of said certain kind of 
materials of said one of said applications into said format with which said requesting consuming device can 
deal, and said step of preparing at least one scenario comprises the step of: 
55 in response to said request for said one of said applications and on the basis of a scenario of said one of said 

applications which scenario is for said input format, preparing a scenario of said one of said applications which 
scenario is adapted to said format with which said requesting consuming device can deal. 
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FIG. 3 
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FIG. 5 
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FIG. 10 
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FIG. 12 
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FIG. 15 
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FIG. 16 

SERVICE OPERATION (SCHEME 3) 



( START ) 



1 


101a 


READ THE SCENARIO DATA FROM THE STORAGE DEVICE 11 BY 
USING A TITLE ID SPECIFIED BY THE REQUESTING CLIENT 3. 
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CONVERT THE MATERIAL FORMAT OF EACH OF THE 
MATERIALS IN THE READ SCENARIO DATA INTO 
THE ONE SPECIFIED BY THE CLIENT 3. 
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REWRITE THE MATERIAL IDs (Sxxxx) WITH MATERIAL 
INFORMATION AND MatlDinVSC's IN THE SCENARIO DATA. 
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CONVERT SCENARIO DATA DEPENDENT ON THE MATERIAL 
FORMAT ACCORDING TO THE SCENARIO CONVERSION TABLE. 
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SEND THE SCENARIO AND THE MATERIALS USED IN 
THE SCENARIO TO THE REQUESTING CLIENT 3. 











21 



v } r 



EP 0 864 990 A2 



FIG. 18 



REGISTRATION OPERATION (SCHEME 4) 
( START ) 
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ASSIGN THE RECEIVED SCENARIO A TITLE ID ; ASSIGN THE 
RECEIVED MATERIALS RESPECTIVE MATERIAL GROUP IDs ; AND 
STORE THE MATERIALS ASSOCIATING THE MATERIALS WITH 
RESPECTIVE MATERIAL GROUP IDs AND MATERIAL FORMATS. 
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FIG. 19 
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FIG. 21 
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CONVERT THE MATERIAL FORMAT OF EACH OF THE 
MATERIALS IN THE READ SCENARIO DATA INTO 
THE ONE SPECIFIED BY THE CLIENT 3. 
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REWRITE THE MATERIAL GROUP IDs (Sxxxx) WITH MATERIAL 
INFORMATION AND MatlDinVSC' s IN THE SCENARIO DATA. 
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CONVERT SCENARIO DATA DEPENDENT ON THE MATERIAL 
FORMAT ACCORDING TO THE SCENARIO CONVERSION TABLE. 
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THE SCENARIO TO THE REQUESTING CLIENT 3. 
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FIG. 22 
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